Papers with visual question answering task
Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA (2021.naacl-main)
Copied to clipboard
| Challenge: | Recent studies show that supervised models exploit data artifacts to achieve good test scores while their performance severely degrades on samples outside their training distribution. |
| Approach: | They propose a method which automatically generates contrast sets for the visual question answering task by using a semantic input representation. |
| Outcome: | The proposed method computes the answer of perturbed questions, thus reducing annotation cost and enabling thorough evaluation of models’ performance on various semantic aspects. |
xGQA: Cross-Lingual Visual Question Answering (2022.findings-acl)
Copied to clipboard
Jonas Pfeiffer, Gregor Geigle, Aishwarya Kamath, Jan-Martin Steitz, Stefan Roth, Ivan Vulić, Iryna Gurevych
| Challenge: | a lack of multilingual multimodal datasets has hindered multimodal vision and language modeling efforts. |
| Approach: | They propose a multilingual evaluation benchmark for the visual question answering task . they extend the established English GQA dataset to 7 typologically diverse languages . |
| Outcome: | The proposed methods outperform current state-of-the-art models in zero-shot cross-lingual settings, but the accuracy remains low across languages. |
A Cognitive Evaluation Benchmark of Image Reasoning and Description for Large Vision-Language Models (2025.naacl-long)
Copied to clipboard
| Challenge: | Large Vision-Language Models (LVLMs) are hardly comprehensively evaluated for their cognitive abilities. |
| Approach: | They propose to evaluate high-level cognitive abilities of Large Vision-Language Models (LVLMs) using images with rich semantics. |
| Outcome: | The proposed evaluation benchmark consists of 251 images along with comprehensive annotations. |
CLIP Models are Few-Shot Learners: Empirical Studies on VQA and Visual Entailment (2022.acl-long)
Copied to clipboard
| Challenge: | Previously, CLIP was only regarded as a powerful visual encoder. |
| Approach: | They propose a parameter-efficient fine-tuning strategy to boost CLIP's few-shot performance on a visual entailment task without introducing any additional pre-training procedure. |
| Outcome: | The proposed strategy achieves competitive zero/few-shot results on visual question answering and visual entailment tasks without introducing any additional pre-training procedure. |
Can VLMs Recall Factual Associations From Visual References? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | a systematic deficiency in the multimodal grounding of Vision Language Models is identified . VLMs can recall factual associations when provided a textual reference to an entity . |
| Approach: | They identify a systematic deficiency in the multimodal grounding of Vision Language Models . they show that VLMs struggle to link their internal knowledge of an entity with its image representation . |
| Outcome: | The study shows that VLMs struggle to link internal knowledge with image representations . the findings provide recommendations for future research . |